Light projector using an acousto-optical control device

ABSTRACT

An approach for projecting light may be implemented using a acousto-optical depth switch that uses surface acoustic waves produced along a substrate to guide image light to different areas. The surface acoustic waves may be generated on a substrate using a transducer. Surface acoustic waves of different frequencies can guide image light onto different optical elements at different physical positions. The optical elements may be configured to show objects in an image at different distances from a viewer.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 14/983,388, filed Dec. 29, 2015, and also claims the benefit ofU.S. Provisional Application Ser. No. 62/097,563, filed on Dec. 29,2014, titled “Acousto-Optical Control Devices for Near-Eye Displays”,the content of the aforementioned application is hereby incorporated byreference in its entirety.

BACKGROUND

Modern computing and display technologies have facilitated thedevelopment of systems for so-called “virtual reality” or “augmentedreality” experiences, wherein digitally reproduced images or portionsthereof are presented to a user and may be perceived as real. A virtualreality (“VR”) scenario typically involves presentation of digital orvirtual image information without transparency to other actualreal-world visual input. An augmented reality (“AR”) scenario typicallyinvolves presentation of digital or virtual image information as anaugmentation to visualization of the actual world around the user. Forexample, referring to FIG. 1, an augmented reality scene 100 is depictedwherein a user of an AR technology device sees a real-world park-likesetting 102 featuring people, trees, buildings in the background, and aconcrete platform 104. In addition to these items, the user of the ARtechnology also perceives that he/she “sees” a robot statue 106 standingupon the real-world platform 104, and a cartoon-like avatar character108 flying by, even though these elements (106, 108) do not exist in thereal world. As it turns out, the human visual perception system is verycomplex, and producing a VR or AR technology that facilitates acomfortable, natural-feeling, rich presentation of virtual imageelements amongst other virtual or real-world imagery elements ischallenging.

Referring to FIG. 2, stereoscopic wearable glasses 200 typeconfigurations have been developed which generally feature two displays(e.g., 202, 204) that are configured to display images with slightlydifferent element presentation such that a three-dimensional perspectiveis perceived by the human visual system. Such configurations have beenfound to be uncomfortable for many users due to a mismatch betweenvergence and accommodation that must be overcome to perceive the imagesin three dimensions. Indeed, some users are not able to toleratestereoscopic configurations.

Referring to FIG. 3, a simplified cross-sectional view of a human eye300 is depicted featuring a cornea 302, iris 304, lens—or “crystallinelens” 306, sclera 308, choroid layer 310, macula 312, retina 314, andoptic nerve pathway 316 to the brain. The macula is the center of theretina, which is utilized to see moderate detail. At the center of themacula is the “fovea”, which is used for seeing the finest details. Thefovea contains more photoreceptors (approximately 120 cones per visualdegree) than any other portion of the retina.

The human visual system is not a passive sensor type of system. It isconfigured to actively scan the environment. In a manner somewhat akinto scanning an image with a flatbed scanner or using a finger to readBraille from a paper, the photoreceptors of the eye fire in response tochanges in stimulation, rather than constantly responding to a constantstate of stimulation. Indeed, experiments with substances such as cobravenom, which is utilized to paralyze the muscles of the eye, have shownthat a human subject will experience blindness if positioned withhis/her eyes open, viewing a static scene with venom-induced paralysisof the eyes. In other words, without changes in stimulation, thephotoreceptors don't provide input to the brain and blindness isexperienced. It is believed that this is at least one reason that theeyes of normal humans have been observed to move back and forth, ordither, in side-to-side motion in what are called “microsaccades”. Asnoted above, the fovea of the retina contains the greatest density ofphotoreceptors, and while humans typically have the perception that theyhave high-resolution visualization capabilities throughout their fieldof view, they generally actually have only a small high-resolutioncenter that they are mechanically sweeping around a lot, along with apersistent memory of the high-resolution information recently capturedwith the fovea. In a somewhat similar manner, the focal distance controlmechanism of the eye (ciliary muscles operatively coupled to thecrystalline lens in a manner wherein ciliary relaxation causes tautciliary connective fibers to flatten out the lens for more distant focallengths; ciliary contraction causes loose ciliary connective fibers,which allow the lens to assume a more rounded geometry for more close-infocal lengths) dithers back and forth by approximately ¼ to ½ diopter tocyclically induce a small amount of what is called “dioptric blur” onboth the close side and far side of the targeted focal length. This isutilized by the accommodation control circuits of the brain as cyclicalnegative feedback that helps to constantly correct course and keep theretinal image of a fixated object approximately in focus.

The visualization center of the brain also gains valuable perceptioninformation from the motion of both eyes and components thereof relativeto each other. Vergence movements (i.e., rolling movements of the pupilstoward or away from each other to converge the lines of sight of theeyes to fixate upon an object) of the two eyes relative to each otherare closely associated with focusing (or “accommodation”) of the lensesof the eyes. Under normal conditions, changing the focus of the lensesof the eyes, or accommodating the eyes, to focus upon an object at adifferent distance will automatically cause a matching change invergence to the same distance, under a relationship known as the“accommodation-vergence reflex.” Likewise, a change in vergence willtrigger a matching change in accommodation, under normal conditions.Working against this reflex, as do most conventional stereoscopic AR orVR configurations, is known to produce eye fatigue, headaches, or otherforms of discomfort in users.

Movement of the head, which houses the eyes, also has a key impact uponvisualization of objects. Humans move their heads to visualize the worldaround them. They often are in a fairly constant state of repositioningand reorienting the head relative to an object of interest. Further,most people prefer to move their heads when their eye gaze needs to movemore than about 20 degrees off center to focus on a particular object(i.e., people don't typically like to look at things “from the corner ofthe eye”). Humans also typically scan or move their heads in relation tosounds—to improve audio signal capture and utilize the geometry of theears relative to the head. The human visual system gains powerful depthcues from what is called “head motion parallax”, which is related to therelative motion of objects at different distances as a function of headmotion and eye vergence distance (i.e., if a person moves his head fromside to side and maintains fixation on an object, items farther out fromthat object will move in the same direction as the head; items in frontof that object will move opposite the head motion. These are verysalient cues for where things are spatially in the environment relativeto the person—perhaps as powerful as stereopsis). Head motion also isutilized to look around objects, of course.

Further, head and eye motion are coordinated with the “vestibulo-ocularreflex”, which stabilizes image information relative to the retinaduring head rotations, thus keeping the object image informationapproximately centered on the retina. In response to a head rotation,the eyes are reflexively and proportionately rotated in the oppositedirection to maintain stable fixation on an object. As a result of thiscompensatory relationship, many humans can read a book while shakingtheir head back and forth (interestingly, if the book is panned back andforth at the same speed with the head approximately stationary, the samegenerally is not true—the person is not likely to be able to read themoving book; the vestibulo-ocular reflex is one of head and eye motioncoordination, generally not developed for hand motion). This paradigmmay be important for augmented reality systems, because head motions ofthe user may be associated relatively directly with eye motions, and thesystem preferably will be ready to work with this relationship.

Indeed, given these various relationships, when placing digital content(e.g., 3-D content such as a virtual chandelier object presented toaugment a real-world view of a room; or 2-D content such as aplanar/flat virtual oil painting object presented to augment areal-world view of a room), design choices may be made to controlbehavior of the objects. For example, the 2-D oil painting object may behead-centric, in which case the object moves around along with theuser's head (e.g., as in a GoogleGlass approach); or the object may beworld-centric, in which case it may be presented as though it is part ofthe real world coordinate system, so that the user may move his head oreyes without moving the position of the object relative to the realworld.

Thus when placing virtual content into the augmented reality world, howthe content is presented must be given consideration. For example, in aworld centric scheme the virtual object stays in position in the realworld so that the user may move his/her ahead around it to see theobject from different points of view.

The systems and techniques described herein are configured to work withthe visual configuration of the typical human to address thesechallenges.

SUMMARY

In some embodiments, an approach for projecting light may be implementedusing a acousto-optical depth switch that uses surface acoustic wavesproduced along a substrate to guide image light to different areas. Thesurface acoustic waves may be generated on a substrate using atransducer. Surface acoustic waves of different frequencies can guideimage light onto different optical elements at different physicalpositions. The optical elements may be configured to show objects in animage at different distances from a viewer.

Further details of aspects, objects, and advantages of some embodimentsare described below in the detailed description, drawings, and claims.Both the foregoing general description and the following detaileddescription are exemplary and explanatory, and are not intended to belimiting as to the scope of the embodiments.

BRIEF DESCRIPTION OF DRAWINGS

The drawings illustrate the design and utility of some embodiments ofthe present invention. It should be noted that the figures are not drawnto scale and that elements of similar structures or functions arerepresented by like reference numerals throughout the figures. In orderto better appreciate how to obtain the above-recited and otheradvantages and objects of various embodiments of the invention, a moredetailed description of the present inventions briefly described abovewill be rendered by reference to specific embodiments thereof, which areillustrated in the accompanying drawings. Understanding that thesedrawings depict only typical embodiments of the invention and are nottherefore to be considered limiting of its scope, the invention will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings in which:

FIG. 1 illustrates an example virtual or augmented reality environment,as according to some embodiments.

FIG. 2 illustrates a virtual or augmented reality headset, as accordingto some embodiments.

FIG. 3. illustrates components of a human eye.

FIG. 4 illustrates a virtual or augmented reality headset and displaymodules, as according to some embodiments.

FIG. 5 illustrates an architecture for a virtual or augmented realityheadset and display modules using a fiber scanning device, as accordingto some embodiments.

FIG. 6 illustrates an example of a virtual or augmented realityenvironment as a flat image, as according to some embodiments.

FIG. 7 illustrate an example of the a virtual or augmented realityenvironment of FIG. 6 split into different depth planes, as according tosome embodiments.

FIG. 8 illustrates an architecture for a virtual or augmented realityheadset and display modules using a fiber scanning device and anacousto-optical depth switch, as according to some embodiments.

FIG. 9 illustrates internal architecture of the acousto-optical depthswitch and a diffractive optical assembly, as according to someembodiments.

FIG. 10 illustrates an architecture for a virtual or augmented realityheadset and display modules using a acousto-optical depth switchdirectly coupled to display circuitry comprising a light generator, asaccording to some embodiments.

FIG. 11 illustrates internal architecture of a diffractive opticalassembly and acousto-optical depth switch having horizontal and verticaltransducers, as according to some embodiments.

FIG. 12 illustrates internal architecture of a diffractive opticalassembly and a horizontal orientated acousto-optical depth switchcoupled to a vertical oriented acousto-optical depth switch, asaccording to some embodiments.

FIG. 13 illustrates internal architecture of a diffractive opticalassembly and a horizontal orientated acousto-optical depth switch inparallel to a vertical oriented acousto-optical depth switch, asaccording to some embodiments.

FIG. 14 illustrates internal architecture of a diffractive opticalassembly and a hybrid fiber scanning and acousto-optical depth switchdevice, as according to some embodiments.

FIG. 15 illustrates internal architecture of a diffractive opticalassembly and a acousto-optical depth switch that covers resolutions thatthe fiber scanning device cannot reach, as according to someembodiments.

FIG. 16A-16C shows flowcharts for methods for projecting light using anacousto-optical depth switch, as according to some embodiments.

FIG. 17 illustrates example system architecture.

DETAILED DESCRIPTION

Various embodiments are directed to a method, system, and computerprogram product for acousto-optical control devices. Other objects,features, and advantages are described in the detailed description,figures, and claims.

Various embodiments of the methods, systems, and articles of manufacturewill now be described in detail with reference to the drawings, whichare provided as illustrative examples so as to enable those skilled inthe art to practice the various embodiments. Notably, the figures andthe examples below are not meant to limit the scope of the presentinvention. Where certain elements of the present invention can bepartially or fully implemented using known components (or methods orprocesses), only those portions of such known components (or methods orprocesses) that are necessary for an understanding of the presentinvention will be described, and the detailed descriptions of otherportions of such known components (or methods or processes) will beomitted so as not to obscure the invention. Further, the presentinvention encompasses present and future known equivalents to thecomponents referred to herein by way of illustration

FIG. 4 illustrates an example system and operating environment in whichthe acousto-optical control devices may be implemented. As shown in FIG.4, an AR system user 400 is depicted wearing a frame 404 structurecoupled to a display system 402 positioned in front of the eyes of theuser. A speaker 406 is coupled to the frame 404 in the depictedconfiguration and positioned adjacent the ear canal of the user (in oneembodiment, another speaker, not shown, is positioned adjacent the otherear canal of the user to provide for stereo/shapeable sound control).The display 402 is operatively coupled 408, such as by a wired lead orwireless connectivity, to a local processing and data module 410 whichmay be mounted in a variety of configurations, such as fixedly attachedto the frame 404, according to some embodiments. In additionalembodiments, the local processing and data module 410 may be fixedlyattached to a helmet or hat, embedded in headphones, removably attachedto the torso of the user (in a backpack-style configuration, orremovably attached to the hip of the user in a belt-coupling styleconfiguration (not depicted).

The local processing and data module 410 may comprise a power-efficientprocessor or controller, as well as digital memory, such as flashmemory, both of which may be utilized to assist in the processing,caching, and storage of data (a) captured from sensors which may beoperatively coupled to the frame 404, such as image capture devices(such as cameras), microphones, inertial measurement units,accelerometers, compasses, GPS units, radio devices, and/or gyros;and/or (b) acquired and/or processed using the remote processing module412 and/or remote data repository 414, possibly for passage to thedisplay 402 after such processing or retrieval.

The local processing and data module 410 may be operatively coupled(416, 418), such as via a wired or wireless communication links, to theremote processing module 412 and remote data repository 414 such thatthese remote modules (412, 414) are operatively coupled to each otherand available as resources to the local processing and data module 410.In some embodiments, the remote processing module 412 may comprise oneor more relatively powerful processors or controllers configured toanalyze and process data and/or image information. In some embodiments,the remote data repository 414 may comprise a relatively large-scaledigital data storage facility, which may be available through theInternet or other networking configuration in a “cloud” resourceconfiguration. In some embodiments, all data is stored and allcomputation is performed in the local processing and data module,allowing fully autonomous use from any remote modules.

FIG. 5 illustrates an example AR system that uses stacked waveguideassemblies (“EDGE”), according to some embodiments. The EDGE system 500generally includes an image generating processor 502, with a memory 512,a CPU 516 and a GPU 514 and other circuitry for image generating andprocessing. The image generating processor 502 may be programmed withthe desired virtual content for presentation to the AR system user. Itshould be appreciated that in some embodiments, the image generatingprocessor 502 may be housed in the wearable AR system. In otherembodiments, the image generating processor and other circuitry may behoused in a belt pack that is coupled to the wearable optics, or otherconfigurations.

The virtual content or information generated by the image generatingprocessor 502 may be transmitted to display circuitry 510. The displaycircuitry 510 may comprise interface circuitry 532 that may be incommunication with the image generation processor 502, and may furtherinterface with circuitry such as chip 534, a temperature sensor 536, apiezoelectric drive/transducer 538, a red laser 540, a blue laser 542,and a green laser 544, and a fiber combiner that combines the lasers(not depicted). Though lasers are illustrated here as an example of alight generator, other types of light generators (e.g., DLP, LCD, LEDs)can also be implemented in display circuitry 510.

The display circuitry 510 may interface with a display or projectivedevice, such as a fiber scanning device (FSD) 520. Generally, an FSD 520is a display device with one or more optical fibers that are vibratedrapidly to create various patterns to deliver the image. More detailsabout the functioning of FSDs are described in U.S. patent applicationSer. No. 14/555,585 filed on Nov. 27, 2014 and entitled “Virtual andaugmented reality systems and methods”, the content of theaforementioned U.S. patent application is hereby expressly incorporatedby reference for all purposes. Although the illustrated embodiment usesan FSD as a display device, one of ordinary skill in the art appreciatesthat other display devices known in the art, (e.g. DLP, OLED, LCDs,LCOS) may be similarly implemented.

The AR system may then use a coupling optic 522 to direct light from theFSD to a diffractive optical element (DOE) assembly 530 (e.g.,diffractive optical elements). The coupling optics 522, according tosome embodiments, may refer to one more lenses that may be used to focuslight to different depth planes in the DOE assembly. Briefly, accordingto some embodiments, a DOE assembly 530 is an apparatus comprised of oneor more stacked planar waveguides with diffraction gratings that (1)deflect the image light along the span of the waveguide, (2) allow theimage light to exit the waveguide at angles that mimic naturalreal-world diffractive effects. Each DOE layer may be customized to aspecific focus depth, as described in further detail below.

FIG. 6 shows an illustrative example of a scene with objects atdifferent distances shown in the same depth plane. There, a flat image600 shows a man 602, a tree 604 which is rooted in the ground 606, and amoon 608 in the sky. In the real world, light diffracts or spreads outas it travels. Thus, light reflected from far away objects, such as themoon 608, has spread out more than light reflected from closer objects,such as the man 602. As explained above, the human vision system handleslight coming from far and near objects in at least two ways (1) by lineof sight adjustments (e.g. vergence movements), and (2) by focusing. Forinstance, when viewing the moon in the real world, the eyes adjust byconverging each eye's line of sight to cross where the moon is located.Similarly, if one stares at the tip of his/her own nose, the eyes willagain adjust converging each eye's line of sight to cross where the tipof the nose is located and the subject will outwardly appear“cross-eyed”.

In addition to adjusting lines of sight, each eye must focus its lensingsystem to account for the spreading out of light. For instance, thelight reflected from the far-away moon 608 may appear more “blurry” thanthe light reflected from the man 602 if the light from the moon is notfocused. Accordingly, to view the moon, each eye focuses its lens byflattening it out to refract the moonlight less and less, which willeventually bring the moon into focus. Similarly, to view the man eacheye focuses its lens by making it more round to increasingly refract theincident light until the man comes into focus. As explained above,adjusting each eye's line of sight and focusing occur togetherautomatically and is known as the “accommodation-vergence reflex.”

The issue with conventional/legacy stereoscopic AR or VR configurationsis that they work against the accommodation-vergence reflex. Forexample, referring to the flat image 600 in FIG. 6, if aconventional/legacy stereoscopic AR or VR system displays the moon 608,the tree 604, and the man 602 at different perceived distances (e.g. theman appears closer and the moon appears farther), but all in-focus, thenthe eyes do not need to refocus when looking at the moon or the man.This causes a mismatch that works against the accommodation-vergencereflex. As mentioned, these sorts of legacy approaches are known toproduce eye fatigue, headaches, or other forms of discomfort in users.

In contrast, the DOE assembly 530 (in FIG. 5) works with the humanaccommodation-vergence reflex by displaying near and far away objects indifferent depth planes. For example, FIG. 7 shows the same flat image600 (e.g. the man, the tree, the ground, and the moon) broken up intothree depth planes, DP1, DP2, DP3, to form a depth composite image 710.The object that is intended to be closest, the man 620, is displayed indepth plane 1 (DP1), which has been tuned to mimic light spreading outfrom objects 1 meter away. The middle objects, the tree 604 and theground 606, are displayed in depth plane 2 (DP2), which has been tunedto mimic light spreading out from objects 5 meters away. Finally, thefarthest object, the moon 608, is displayed in depth plane 3 (DP3),which has been tuned to mimic light spreading out from objects384,400,000 meters away. (384,400,000 meters is the approximate distancefrom the Earth to the Moon. However, for objects past a certain distanceit is common to simply adjust the imaging system, such as a lensingsystem, to optical infinity, whereby the incident light rays areapproximated as nearly parallel light rays.) In this way, a viewer ofthe depth-composite image 710 must adjust both his/her focusing and lineof sight convergence when looking at the objects in the different depthplanes, and no headaches or discomfort will occur.

Referring again to FIG. 5, the image generating processor 502 may beimplemented as the device that “breaks-up” a flat image into a number ofobjects in a number of depth planes, according to some embodiments. Inother embodiments, the image sequence is stored as separate depth planespecific image sequences, and the image processing generator transmitsthe pre-processed depth plane image sequences to the display circuitryready for display.

In some embodiments, the DOEs are switchable between “on” states inwhich they actively diffract, and “off” states in which they do notsignificantly diffract. For instance, a switchable DOE may comprise alayer of polymer dispersed liquid crystal, in which microdropletscomprise a diffraction pattern in a host medium, and the refractiveindex of the microdroplets can be switched to substantially match therefractive index of the host material (in which case the pattern doesnot appreciably diffract incident light) or the microdroplet can beswitched to an index that does not match that of the host medium (inwhich case the pattern actively diffracts incident light). More detailsabout the volume phase holograms are described in U.S. patentapplication Ser. No. 14/555,585 filed on Nov. 27, 2014 and entitled“Virtual and augmented reality systems and methods”, the content of theaforementioned U.S. patent application is hereby expressly incorporatedby reference for all purposes.

To conserve resources, such as battery power, in some embodiments it maybe preferable to only display image information for a certain depthplane when the viewer is looking at objects in the depth plane. Forinstance, referring to FIG. 7, if the image consists only of the moon608, then DP3 may be switched on, while the others depth planes, DP1 andDP2 switched off. Optionally, all three depth planes may be turned onand used to display objects in a sequenced fashion. For example, the FSD520 may quickly switch between projecting images on DP1, DP2, DP3 inrapid succession. Because the human vision system can only detectmovements/changes up to a certain frequency (e.g. 30 Hz), the viewerwill not perceive that the FSD 520 is switching between planes but willinstead perceive a smooth multi-depth planed composite image stream.

Additionally, according to some embodiments, the system may also includean eye-tracking subsystem 550 (FIG. 5). In this case, the eye-trackingsubsystem can monitor the viewer's eye's (for instance by monitoring theeye's convergence angles) to determine whether the viewer is looking ata far object or a close object. If the system detects that the viewer islooking at the moon, for instance, then DP3 can be switched on, and DP1and DP2 switched off and/or attenuated.

A stacked configuration may use dynamic DOEs (rather than staticwaveguides and lenses) to provide multiplanar focusing simultaneously.For example, with three simultaneous focal planes, a primary focus plane(based upon measured eye accommodation, for example) could be presentedto the user, and a + margin and − margin (one focal plane closer, onefarther out) could be utilized to provide a large focal range in whichthe user can accommodate before the planes need be updated. Thisincreased focal range can provide a temporal advantage if the userswitches to a closer or farther focus (e.g., as determined byaccommodation measurement). Then the new plane of focus could be made tobe the middle depth of focus, with the + and − margins again ready for afast switchover to either one while the system catches up.

However, this scenario assumes that the FSD is able to operate fastenough to rapidly generate different images/portions of the images to beinjected into multiple DOEs. As explained, FSDs generally work byrastering back and forth over a given angle. The angle dictates thefield of view (FOV) for the image that is displayed. In a system withsix depth planes (e.g. DP1, DP2 . . . DP6), the FSD must be able toswitch between depth planes six times per frame in a seamless manner.For example, if the frames per second (FPS) is 60 (typical in many videostream implementations), then for each frame the FSD must switch sixtimes per frame. Additionally, in each depth plane there may be twotarget zones, one for green light and a second one for red and bluelight. Accordingly, there may be 12 targets per frame that the FSD mustbe able to switch to. Thus, for 60 FPS and 12 targets the FSD must beable to switch approximately 714 times per second to raster a seamlessimage/video sequence. Because a FSD is a physical/mechanical device thatactuates a fiber through an angle to raster images, it becomesincreasingly difficult to actuate over larger angles fast enough, as theframes per second or number of depth planes increases.

Additionally, assuming FSD 520 can raster and switch fast enough, thecoupling optics 522 (which direct light received from the FSD into theDOE assembly at nearly orthogonal angles) should be capable of matchingthe speed and FOV requirements of the FSD. Current approaches, such asusing lenses to focus FSD light onto each depth plane, are limited atleast with respect to the FOV requirements. Ideally, for realisticsimulations, an FOV of 120 degrees is required to mimic naturalreal-world vision. However, current coupling optic approaches, such asusing a variable focus lensing system, LC shutters, and/or gratingsystems, cannot product 120 degrees FOV, and cannot switch between depthplanes fast enough to produce a seamless visual display.

Additionally, mechanically actuating an FSD and coupling optics, such asa lensing system, can drain power and resources, even if such approachescould switch fast enough over the required FOV. Thus, there is a needfor an approach for quickly displaying images in multiple depth planesover a large field of view.

FIG. 8 illustrates an approach for quickly displaying images in multipledepth planes over a large field of view. There, the architecture 800 issimilar to the architecture illustrated in FIG. 5, with exception to theacousto-optical depth switch (ADS) 802 that is capable of matchingand/or exceeding the FSD's speed over a large FOV, such as 120 degrees.As illustrated in the example embodiment of FIG. 8, the ADS 802 iscoupled to receive light from the FSD 520 and focus the light ontodifferent DOE layers that are at different depths.

FIG. 9 illustrates internal architecture 900 showing aspects of the ADSand the DOE assembly, as according to some embodiments. There, the ADS802 includes a logic module 950 and an acousto-optical (AO) modulator952. In the embodiment illustrated, the light input 902 from the FSD 520enters the ADS 802 unit and is deflected (e.g. diffracted, refracted) ata number of angles into the the DOE assembly 530. Each DOE layer ordiffractive element (e.g. 530 a, 530 b, 530 c) corresponds to a depthplane (e.g. DP1, DP2, DP3). For example, DOE layer 530 a may correspondto DP1, and displays the man 620 (FIG. 7) at a perceived distance of 1meter away from the viewer. Likewise, DOE layer 530 b may correspond toDP2, and displays the tree 604 rooted in the ground 606 at a perceiveddistance of 5 meters away from the viewer. Finally, DOE layer 530 c maycorrespond to DP3, and displays the moon 608 at a perceived distance of384,400,000 meters away (or at optical infinity).

In some embodiments, each DOE layer implements an in-coupling grating960 to deflect the image light received form the ADS 802 along the spanof the depth plane. The image may then exit the DOE layers towards theviewer 914 using a second set of diffraction gratings (not depicted).More details about the gratings are described in U.S. patent applicationSer. No. 14/555,585, filed on Nov. 27, 2014 and entitled “Virtual andaugmented reality systems and methods”, as well as U.S. patentapplication Ser. No. 14/726,424, filed on May 29, 2015 and entitled“Methods and systems for virtual and augmented reality”. The content ofthe aforementioned U.S. patent application is hereby expresslyincorporated by reference for all purposes.

In some embodiments, the AO modulator receives the light through acoupling optic, guides the received light along a waveguide, uses atransducer to cause surface acoustic waves along a substrate (thesurface acoustic waves change the index of refraction of the substrate),which causes the light to exit the substrate at an angle proportional tothe surface acoustic wave period. In particular, as illustrated in FIG.9, the input light 902 first interfaces with the AO modulator 952through a coupler 904, such as a prism. The coupler 904 directs thelight into a waveguide 906 on a substrate 912. In some embodiments, thesubstrate comprises a piezoelectric material such as quartz, or otherpiezoelectric transparent/translucent materials as are known in the art.In some embodiments, the substrate comprises a thin sheet of lithiumniobate, which is also piezoelectric (i.e., generates electricity inresponse to pressure/stress).

In some embodiments, the lithium niobate substrate may be used as anelectro-optical switch by applying high voltages (e.g. 30 volts) tochange the index of refraction of the material and refract light indesired directions. However, running high voltages near the human faceis typically not desired. Further, using high voltage switches, such asa 30-volt lithium niobate switch, may not be practical in wearablecomputer-vision systems where battery power is typically limited.

Alternatively, as illustrated in FIG. 9, instead of using the substrateas an electro-optical switch, the AO modulator uses the substrate 912 asan acousto-optical switch. For example, a transducer 908 may be suppliedwith very low voltages that causes the substrate to jiggle back andforth to produce waves along the surface of the substrate (e.g. “surfaceacoustic waves”). The surface acoustic waves may have a certain definedperiod (e.g. the distance from peak-to-peak) that is proportional to thefrequency of waves produced by the transducer. That is, for example, ifthe transducer 908 receives 60 Hz AC, the period of the surface acousticwaves approximately matches 60 Hz (discounting, for example, the energylost in the material itself, e.g., hysteresis). Likewise, if RFfrequency power is supplied to the transducer, the surface acousticwaves will approximately match the RF frequencies. Thus, by changing thefrequency of the transducer, the period of the induced surface waves canbe controlled and/or tuned. Generally, in some embodiments, the logicmodule 950 may manage the AO modulator 952 to produce the requiredfrequencies. For example, the logic module may receive a stream of datacauses the transducer to change frequencies in a sequence to directlight to the DOE assemble layers. In other embodiments, othercomponents, such as the image processing generator 502, manage the AOmodulator to produce the sequences of frequencies.

As mentioned, the surface acoustic waves change the index of refractionof the substrate and may also act as a type of diffraction grating.Initially, the waveguide and the substrate have two different indices ofrefraction, such that total internal reflection occurs for light insidethe waveguide. Certain substrates, such as lithium niobate, have anindex of refraction that changes in response to electrical energy orphysical/mechanical energy (e.g. stresses). As such, by applyingdifferent surface acoustic waves to a lithium niobate substrate, theindex refraction can be changed so as to breakdown the total internalreflection occurring within the waveguide and thus allow the lightinside the waveguide to escape.

Further, the angle at which light of a given wavelength is deflected outof a grating is proportional to the wavelength of the light. Forexample, shining white light on a grating yields rainbows of “broken-up”colors that correspond to different wavelengths. In some embodiments,the surface acoustic waves act as a diffraction grating that diffractsthe image light out of the waveguide/substrate interface (e.g. theinterface between 912 and 906 in FIG. 9) at angles proportional to thegrating width (e.g. the distance from peak to peak for the surfaceacoustic wave). In this way, the input light 902 traveling through thewaveguide 906 may be deflected by refraction (caused by the change inindex of refraction of the substrate 912) and diffraction (caused by thesurface acoustic waves inducing a diffraction grating effectproportional to the wave period). The combined effects can be used toguide the input light 902 onto a number of in-coupling grating targets,such as in-coupling grating 906. Additionally, the speed at which lightcan be deflected from one target to the next can be adjusted by simplyapplying a different signal (e.g. different frequency) to the transducer908. In this way, the acousto-optical depth switch 802 can attain veryhigh switching speeds over a large FOV.

FIG. 10 illustrates an approach that uses an acousto-optical device as ascanner and switch, without the need for a FSD and/or coupling optic.There, the architecture 1000 is similar to the architecture illustratedin FIG. 8, with exception to the acousto-optical scanner (AOS) 1002 andlack of FSD 520. In operation, image signal from the display circuitry510 is input directly into the AOS 1002. The AOS 1002 may then modulateand deflect the light onto different depth planes using acousto-opticalapproaches like those discussed above.

FIG. 11. illustrates internal architecture 1100 of the acousto-opticalscanner (AOS) 1002 and DOE assembly 530, as according to someembodiments. As illustrated, the input light/signal 902 from the displaycircuit 510 (FIG. 5) may interface first with the coupler 1114, whichmay be an optical coupler such as a prism. The coupler 1114 directs thelight into a waveguide 1110 which uses total internal reflection toguide the light on a substrate 1108. In contrast with the approachesdiscussed above, the AO modulator 1106 in FIG. 11 has two transducers.The vertical transducer 1120 is discussed above, and generally producesvertical surface acoustic waves 1118 that cause the light to deflect atdifferent angles towards the DOE assembly 530.

The horizontal transducer 1116, in some embodiments, may be alignedorthogonal to the vertical transducer 1120. The horizontal transducer isimplemented to produce horizontal surface acoustic waves 1112. Like thevertical surface acoustic waves 1118, which deflect the input lightvertically (relative to the AO modulator), the horizontal surfaceacoustic waves may also deflect light in the waveguide but horizontally,using mechanisms such as Bragg diffraction. Thus as implemented, the AOmodulator 1106 can control the input light in both the horizontal andvertical directions. For example, with reference to image output 1150,in DP2 the image to be displayed is the tree rooted in the ground. Todirect the beam to scan the image horizontally 1152, the horizontaltransducer can modulate the horizontal surface acoustic waves bycontrolling the frequency and thus the horizontal deflection of thelight. Likewise, to scan the image output vertically 1154, the verticaltransducer 1120 can modulate the vertical surface acoustic waves 1118 bycontrolling the frequency and thus the vertical deflection of light.

FIG. 12 shows an AOS architecture 1200 for deflecting the light using ahorizontal AO modulator and a vertical AO modulator in a hybrid AOS unit1202, as according to some embodiments. There, the horizontal AOmodulator 1204 may comprise the coupler, substrate, waveguide, and ahorizontal transducer (e.g., horizontal transducer 1116), which may beused to produce horizontally deflected or shifted light 1222. Thehorizontally deflected light may then be input into the vertical AOmodulator 1206. The vertical AO modulator may comprise a coupler,substrate, waveguide and a vertical transducer (e.g., verticaltransducer 1120) which produces vertical surface acoustic waves thatdeflect the light vertically 1224. Thus instead of one combinedvertical/horizontal AO modulator (e.g., 1106 in FIG. 11), the twomodulators (1204, 1206) are individual units and each may have their ownsubstrate, coupler, and waveguide but with orthogonal transducers.

FIG. 13 shows an AOS architecture 1300 for deflecting the light using anupright modulator and an orthogonal modulator a hybrid AOS unit 1310, asaccording to some embodiments. There, the upright modulator 1320 isconstructed like the AO modulator 952 illustrated in FIG. 9. That is, itis capable of deflecting light in the up/down direction (relative to themodulator). When vertical input light 1304 is input into the uprightmodulator 1320 it is deflected in the vertical direction to scan animage, such as the image output 1150 in the vertical direction 1154.

The orthogonal AO modulator 1322 is also constructed like the AOmodulator 952 illustrated in FIG. 9. However, the orthogonal AOmodulator may be rotated 90 degrees so that it is orthogonal to theupright modulator 1320. In this way, the orthogonal AO modulator 1322deflects horizontal input light 1302 to scan the image in the horizontaldirection 1152, without using Bragg diffraction. Though orthogonalmodulators are illustrated here as an example, one of ordinary skill inthe art appreciates that one or more AO modulators aligned at differentangles may similarly be implemented to achieve full image scans. Forexample, in a three AO modulator implementation, a first AO modulatormay be aligned at 0 degrees and input light into a second AO modulatorwhich is oriented at 45 degrees (relative to the first AO modulator)which may input light into a third AO modulator oriented at 90 degrees(relative to the first AO modulator). In this way, the one or morein-between modulators can lessen slowly change the angles instead ofgoing from 0 to 90 degrees in one step.

In some embodiments, it may be preferable to have one substrate, butwith two of its orthogonal surfaces utilized. For instance, the top faceof the substrate may implement a first coupler, waveguide, andtransducer. While on the side face of the substrate, a second coupler,waveguide and transducer is implemented. In operation, this embodimentfunctions similar to the upright and orthogonal modulators illustratedin FIG. 13 but without the need for a second substrate and/or AOmodulator unit.

FIG. 14 illustrates an architecture 1400 for implementing a hybridFSD/AOS module, as according to some embodiments. There, the hybridFSD/AOS module 1402 is structurally similar to the FSD 520 and ADS 802in FIG. 8. However, in the approach illustrated in FIG. 14, the AOScomponent is used as a complementary scanner/generator and switch. FIG.15 shows internal architecture 1500 of the AOS modulator 1550 asaccording to this embodiment. In this approach, an FSD (e.g., FSD 520)generates an image to be displayed at a certain resolution, the image isinput from the FSD as illustrated at 1504. For example, referring to theimage output 1530, FSDs generally have a limited resolution and canoutput light along a swirl at certain spacings. That is, the swirl 1510in the image output 1530 represents points in which the FSD can projectlight. The circular points 1512 between the swirl are beyond theresolution of the FSD. However, though the FSD cannot reach the circularpoints between the swirl, the AO module can. In this approach, thehybrid FSD/AOS component features an AO modulator 1550 with bothhorizontal and vertical modulators, which can more finely generate imagepoints that the FSD cannot target or reach. As according to someembodiments, the “primary” image points may first be generated by theFSD (e.g. the points along the FSD swirl 1510), whereas thesecondary/complementary image points are then generated by the AOmodulator 1550 so as to “fill-in” the points that lie beyond theresolution of the FSD.

FIG. 16A shows a flowchart 1600 for an approach for projecting lightusing an acousto-optical depth switch, as according to some embodiments.At 1602, an image generator, such as lasers, LEDs, or an LCD, generatesimage light comprising a series of images. The series of images may be avideo sequence of images, where each image in the series depicts objectsat different distances. For example, a first portion of the series couldcomprise all the objects in a first depth plane which is closed toviewer (e.g., viewer wearing a virtual reality or augmented realityheadset). Likewise, other portions of the series may comprise objects atdifferent distances. In an exemplary embodiment, six depth planes areimplemented, each of which corresponding to six distances from theviewer. In some embodiments, the first depth plane of six corresponds toa distance of three meters or closer, and the sixth depth planecorresponds to optical infinity or an otherwise very large distance.

At 1604, the image light generated by the light generator is input intoan FSD, which actuates over an angle. As according to some embodiments,the FSD is used to project the light onto an acousto-optical depthswitch coupling optic as shown at 1606. The coupling optic, such as aprism, may direct the image light onto a wave guide, along a substrate.A transducer within the acousto-optical depth switch may vibrate atdifferent frequencies to generate surface acoustic waves on the surfaceof the substrate. As explained above, surface acoustic waves ofdifferent frequencies deflect the image light at different angles.

At 1608, the transducer may receive instructions from a logic modulethat instructs the transducer to produce SAWs at different frequenciesto deflect the image light onto different optical elements, such asdiffractive optical elements.

FIG. 16B illustrates a flowchart 1609, for using a acousto-optical depthswitch to deflect light at different frequencies, as according to someembodiments. In some embodiments, the image light may be sequences intoportions of light for different depth planes. For example, a firstleading portion may comprise objects that are to be shown as closest tothe viewer. The second portion may comprise objects that to be shown atan intermediate distance to the viewer. A third portion may compriseobjects that are to be shown a farthest distance from the viewer. Alogic module may direct the transducer to product SAWs of differentfrequencies in an alternating fashion to deflect the first portion to afirst optical element using a first frequency as shown at 1610, a secondportion to a second optical element using a second frequency as shown at1612, and a third portion to a third optical element using a thirdfrequency as shown at 1613. Although only three depth planes andfrequencies are discussed here as an example, other numbers of depthplanes (e.g., six) and corresponding frequencies can likewise beimplemented.

FIG. 16C shows a flowchart 1614 for an approach for projecting light inorthogonal directions using orthogonally oriented transducers, asaccording to some embodiments. At 1616, horizontal SAWs are generatedusing a horizontal transducer. The horizontal SAWs can deflect or rasterlight onto an optical element along a horizontal direction using Braggdiffraction. At 1618, vertical SAWs are generated using a verticaltransducer. The vertical SAWs can defect or raster light onto an opticalelement along a vertical direction using refraction and diffraction.

FIG. 17 is a block diagram of an illustrative computing system 1700suitable for implementing a light projector and the logic moduleaspects, as according to some embodiments. Computer system 1700 includesa bus 1706 or other communication mechanism for communicatinginformation, which interconnects subsystems and devices, such asprocessor 1707, system memory 1708 (e.g., RAM), static storage device1709 (e.g., ROM), disk drive 1710 (e.g., magnetic or optical),communication interface 1714 (e.g., modem or Ethernet card), display1711 (e.g., CRT or LCD), input device 1712 (e.g., keyboard), and cursorcontrol.

According to one embodiment of the invention, computer system 1700performs specific operations by processor 1707 executing one or moresequences of one or more instructions contained in system memory 1708.Such instructions may be read into system memory 1708 from anothercomputer readable/usable medium, such as static storage device 1709 ordisk drive 1710. In alternative embodiments, hard-wired circuitry may beused in place of or in combination with software instructions toimplement the invention. Thus, embodiments of the invention are notlimited to any specific combination of hardware circuitry and/orsoftware. In one embodiment, the term “logic” shall mean any combinationof software or hardware that is used to implement all or part of theinvention.

The term tentative embodiments, hard-wired circuitry may be used inplace of refers to any medium that participates in providinginstructions to processor 1707 for execution. Such a medium may takemany forms, including but not limited to, non-volatile media andvolatile media. Non-volatile media includes, for example, optical ormagnetic disks, such as disk drive 1710. Volatile media includes dynamicmemory, such as system memory 1708. According to some embodiments, adatabase 1732 may be accessed on a computer readable medium 1731 using adata interface 1733″

Common forms of computer readable media includes, for example, floppydisk, flexible disk, hard disk, magnetic tape, any other magneticmedium, CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, RAM, PROM, EPROM,FLASH-EPROM, any other memory chip or cartridge, or any other mediumfrom which a computer can read.

In an embodiment of the invention, execution of the sequences ofinstructions to practice the invention is performed by a single computersystem 1700. According to other embodiments of the invention, two ormore computer systems 1700 coupled by communication link 1715 (e.g.,LAN, PTSN, or wireless network) may perform the sequence of instructionsrequired to practice the invention in coordination with one another.

Computer system 1700 may transmit and receive messages, data, andinstructions, including program, i.e., application code, throughcommunication link 1715 and communication interface 1714. Receivedprogram code may be executed by processor 1707 as it is received, and/orstored in disk drive 1710, or other non-volatile storage for laterexecution.

In the foregoing specification, the invention has been described withreference to specific embodiments thereof. It will, however, be evidentthat various modifications and changes may be made thereto withoutdeparting from the broader spirit and scope of the invention. Forexample, the above-described process flows are described with referenceto a particular ordering of process actions. However, the ordering ofmany of the described process actions may be changed without affectingthe scope or operation of the invention. The specification and drawingsare, accordingly, to be regarded in an illustrative rather thanrestrictive sense.

What is claimed is:
 1. A system for projecting light, the systemcomprising: a light generator that generates image light thatcorresponds to a series of images; a display device configured toreceive the image light generated by the light generator and transmitdisplay image light at a first resolution, the display image lightcomprising a first set of image points; and an acousto-optical scanneroptically coupled to the display device, wherein the display devicedirects the display image light onto the acousto-optical scanner and theacousto-optical scanner directs portions of the display image light ontoa plurality of diffractive optical elements, wherein each of theplurality of diffractive optical elements corresponds to a differentdepth plane of the series of images; wherein the acousto-optical scanneris configured to receive the display image light from the display deviceat the first resolution corresponding to the first set of image points,selectively generate a second set of image points, and output imagelight at a second resolution greater than the first resolution, theoutput image light comprising the first set of image points and thesecond set of image points.
 2. The system of claim 1, wherein thedisplay device is further configured to generate the display imagelight.
 3. The system of claim 1, wherein the display device comprises afiber scanning device.
 4. The system of claim 3, wherein the fiberscanning device comprises one or more fibers that actuate over an angleto direct the image light generated by the light generator as thedisplay image light onto the acousto-optical scanner.
 5. The system ofclaim 4, wherein the one or more fibers vibrate rapidly creatingpatterns to direct the image light generated by the light generator asthe display image light onto the acousto-optical scanner.
 6. The systemof claim 4, wherein the one or more fibers raster back and forth over agiven angle.
 7. A system for projecting light, the system comprising: alight generator that generates image light that corresponds to a seriesof images; a display device configured to: receive the image lightgenerated by the light generator, and transmit display image lightcomprising a plurality of image points; and an acousto-optical scannerconfigured to: receive the display image light, in a first mode, directa first set of image points from the plurality of image points onto aplurality of diffractive optical elements, and in a second mode,generate a second set of image points from the plurality of image pointsand direct the second set of image points onto the plurality ofdiffractive optical elements, wherein each of the plurality ofdiffractive optical elements corresponds to a different depth plane ofthe series of images.
 8. The system of claim 1, wherein the displaydevice is further configured to generate the display image light.
 9. Thesystem of claim 1, wherein the display device comprises a fiber scanningdevice.
 10. The system of claim 9, wherein the fiber scanning devicecomprises one or more fibers that actuate over an angle to direct theimage light generated by the light generator as the display image lightonto the acousto-optical scanner.
 11. The system of claim 10, whereinthe one or more fibers vibrate rapidly creating patterns to direct theimage light generated by the light generator as the display image lightonto the acousto-optical scanner.
 12. The system of claim 10, whereinthe one or more fibers raster back and forth over a given angle.